IEEE TRANSACTIONS ON NEURAL NETWORKS VOL XX NO Y MONTH E cient Algorithms for Function Approximation with Piecewise Linear Sigmoidal Networks

This paper presents a computationally e cient algorithm for function approximation with piecewise linear sigmoidal nodes A one hidden layer network is constructed one node at a time using the well known method of tting the residual The task of tting an individual node is accom plished using a new algorithm that searches for the best t by solving a sequence of Quadratic Programming problems This approach o ers signi cant advantages over derivative based search algorithms e g backpropagation and its ex tensions Unique characteristics of this algorithm include nite step convergence a simple stopping criterion solu tions that are independent of initial conditions good scaling properties and a robust numerical implementation Empir ical results are included to illustrate these characteristics